Method and apparatus for coding an input signal based on characteristics of the input signal

ABSTRACT

An interframe coding system which eliminates higher frequency components contained in an image signal effectively and adaptively with an adaptive filter provided in a coding loop. The adaptive filter eliminates the higher frequency components with the optimal filtering intensity for an image signal specified with a filtering coefficient which is decided by a filter controller. The filtering coefficient is decided by normalization of the difference between an input image signal and a predictive signal from a frame memory by the &#34;Activity&#34; of the input image signal or the predictive signal. The &#34;Activity&#34; can be based upon the sum of the absolute or squared difference values based upon the mean value of luminance intensity of pixels of the image signal.

This application is a division of application Ser. No. 08/157,638 filedon Nov. 24, 1993, now abandoned.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to an image coding system, more specifically toan interframe coding system, wherein an image signal is coded on aframe-to-frame difference basis.

2. Discussion of the Related Art

It is a common practice to remove redundant components in an imagesignal for a highly efficient coding. Especially with a dynamic imagesignal, an interframe coding is one of the preferable and typical arts.The interframe coding is a coding technique which codes a differencebetween a newly input uncoded image signal and a previously coded imagesignal.

FIG. 40 is a block diagram of a conventional interframe coding apparatusdisclosed in Japanese Unexamined Patent Publication No. 208382/1988. Theinterframe coding apparatus according to the figure has a frame memory1, a motion-vector detector 2, a subtractor 3, an encoder 4, a localdecoder S, an adder 6, a filter 7 and a filter controller 8. The framememory 1 stores an image signal of the previous frame.

Operation of the conventional interframe coding system with a localdecoding loop is now described with reference to the FIG. 40.

The input image signal 12 is compared with the image signal of theprevious frame 11 stored in the frame memory 1 by a block-matchingtechnique in the motion vector detector 2. The motion vector detector 2detects the quantity and direction of motion of the input image signal12 and outputs a motion vector signal 13. The frame memory 1 outputs amotion compensation predictive signal 14 based upon the motion vectorsignal 13. The subtractor 3 subtracts the motion compensation predictivesignal 14 from the input image signal 12 to output a predictive errorsignal 15 or a difference signal.

The predictive error signal 15 is coded by quantization at the encoder4, and a coded error signal 16 is output. The coded error signal 16 isdecoded in the local decoder 5, and a local decoded error signal 17 isoutput. The local decoded error signal 17 is added to the motioncompensation predictive signal 14 in the adder 6, and a local decodedsignal 18 is output. The local decoded signal 18 is filtered by thefilter 7 to eliminate higher frequency components in the signal. Thefilter smooths the local decoded signal and outputs a smoothed localdecoded signal 19. Filtering is controlled by a control signal 20 outputby the filter controller 8. The filter control signal 20 controls thefilter based upon the motion vector signal 13.

The coded error signal 16 and the motion vector signal 13 aretransmitted via a transmission line to an external decoding system.

Image coding is generally processed by the unit or block of 16×16 or 8×8pixels of an image signal.

With reference to other conventional coding techniques, the filter 7 canbe placed after the frame memory 1 in the local decoding loop instead ofafter the adder 6 as shown in FIG. 40. Further, filtering can beaccomplished by an intra-block filter which processes pixels within ablock, and by an inter-block filter which processes pixels involving thepixels in the neighboring blocks. Furthermore, a motion detection can beachieved with a smaller unit of a pixel than a full pixel. This isdesigned to detect the optimal block of pixels in the previous framewhich match a block of pixels in the input image signal 12. Theseconventional techniques can contribute to eliminate higher frequencycomponents or redundancy in the input image signal based upon thequantity of motion. Thus, they are effective for removing noise in thesignal. Consequently, coding efficiency can be improved greatly withthese conventional arts.

An intraframe coding with a local decoding loop is processed in thefollowing manner according to the conventional coding with reference toFIG. 1. The input image signal 12 is directly coded by quantization inthe encoder 4, where the coded error signal 16 is output. The codederror signal 16 is decoded in the local decoder 5, where the localdecoded error signal 17 is output. The local decoded error signal 17 isstored in the frame memory 1. The coded error signal 16 is transmittedvia a transmission line to an external decoding system.

The conventional interframe coding generally controls filtering basedupon the motion vector, whereby an image signal can be filtered basedupon the quantity of motion. In other words, a block of pixelsrepresenting motion of an image signal, or a motion block, is filteredwith a low-pass filter (LPF) thereby eliminating the original definitionto eliminate noise in the signal. On the other hand, a block of pixelsrepresenting no motion or a very small amount of motion, is notfiltered.

FIG. 41 illustrates filtering different characteristics of the motioncompensation predictive signal 14 using the LPF in relation withluminance intensity (I) versus frequency (f). In the figure, 14adesignates a characteristic of the image signal from the frame memory 1without LPF filtering. 14b designates a characteristic of the imagesignal from the frame memory 1 after LPF filtering.

In other words, a picture with no or little motion is unfiltered to keepits original definition as characteristic 14a shows. An image signal inmotion is filtered with the LPF reducing its original definition toeliminate higher frequency components in the signal as characteristic14b shows. Thus, LPF filtering eliminates higher frequency components,designated by the shaded portion in FIG. 41, in the image signal basedupon the quantity of motion of the image signal.

FIGS. 42(a), 42(b) and 42(c) respectfully illustrate LPF filteringcharacteristics of the input image signal 12, the motion compensationpredictive signal 14a, and the predictive error signal 15. When themotion compensation predictive signal 14a can reproduce the input imagesignal 12 successfully, as shown in the figure, according to theexcellent motion detection performance, the difference or the predictiveerror signal 15 between the two signals is small. Consequently, this cancontribute to a higher coding efficiency because the amount of codinginformation is reduced.

Accordingly, when a motion block of the motion compensation predictivesignal has a high quality of reproduction of the input image signal withthe excellent motion detection performance and the difference betweenthe two signals is very small, the block need not to be filtered.However, this does not apply to some cases: an image signal is filteredirrespective of the reproduction quality of the motion compensationpredictive signal.

When the motion compensation predictive signal 14 is filtered with theLPF, the signal loses higher frequency components as shown in FIG. 41.As a result, the difference between the input image signal and thefiltered motion compensation predictive signal is the higher frequencycomponents. In other words, a large amount of coding is needed therebyreducing the coding efficiency and resulting in a low definition.

FIGS. 43(a), 43(b) and 43(c) respectfully illustrate the characteristicsof two input signals and one output signal at the subtractor 3. FIG.43(a) shows the characteristic of the input image signal 12, whichcorresponds to that of FIG. 42 (a). FIG. 43(b) shows the characteristicof the filtered motion compensation predictive signal 14b in comparisonto that of the unfiltered motion compensation predictive signal 14a.This shows that the higher frequency components (noise) in the signalwith motion is eliminated by the LPF filtering. FIG. 43(c) shows thecharacteristic of the predictive error signal 15 which is the differencebetween the two input signals 12 and 14. The difference still containsthe higher frequency components of the motion compensation predictivesignal 14. This requires the encoder to have to encode bigger amounts ofinformation leading to a lower coding efficiency.

Thus, the problem is stemmed from the unnecessary filtering as FIGS.42(a), 42(b), 42(c), 43(a), 43(b) and 43(c) illustrate. The motioncompensation predictive signal 14 is not to be filtered when the signalhas a high quality of reproduction due to a high detection performanceof motion as shown in FIGS. 42(a), 42(b) and 42(c). When the motioncompensation predictive signal 14 is filtered unnecessarily under thatcondition as shown in FIGS. 43(a), 43(b) and 43(c), the image is damagedinvolving poor coding efficiency due to a higher amount of codinginformation or involving poor definition.

The conventional interframe coding system carries another problem ofquantization. Coding performance, according to the conventional system,is based upon a limited quantization which is designed optimally for acertain pattern of predictive error signal 15. In other words, thelimited quantization can not deal effectively with coding signals ofvarious patterns. When the encoder quantizes the signal with the limitedquantization, the predictive error signal 15 is characterized with apoor coding efficiency and results in producing a poor coded errorsignal.

The present invention is devoted to solve the problems. An objective ofthis invention is to provide an interframe coding system with a highcoding efficiency by eliminating higher frequency components remainingin a coded signal.

Another objective of this invention is to provide a coding controllerwhich allows the encoder to code efficiently with coding signals ofvarious patterns. This can lead to the overall coding efficiency of theinterframe coding system.

SUMMARY OF THE INVENTION

An interframe coding system according to one aspect of the presentinvention for frame-to-frame coding of an input signal, having a localdecoding loop which includes a filter for filtering a predictive signal,may include a filter controller for receiving the input signal and thepredictive signal, and for generating a control signal to control thefilter.

The local decoding loop may includes decoding means for producing aprevious input signal from a previous frame, a motion vector detectorwhich receives the input signal and the previous input signal, andoutputs a motion vector, and a frame memory for receiving the motionvector and the previous input signal and outputting a motioncompensation predictive signal to the filter controller as thepredictive signal.

The filter controller may include a difference calculator for receivingthe input signal and the predictive signal. calculating a differencebetween the input signal and the predictive signal and outputting adifference signal, a decision unit for receiving the difference signaland the input signal, and for outputting the control signal, and whereinthe filter may include a plurality of patterns of filtering coefficientsand a means for selecting a pattern of filtering coefficients based onthe control signal.

The decision unit may include means for calculating an activity valuefrom the input signal, and means for normalizing the difference by theactivity value and for outputting the control signal.

The decision unit may include means for receiving the predictive signalinstead of the input signal, calculating an activity value from thepredictive signal, and means for normalizing the difference by theactivity value and for outputting the control signal.

The decision unit may include means for comparing the difference signaland a predefined value, and issuing the control signal based on acomparison result.

The filter controller may include means for calculating a characteristicof the input signal, comparing the characteristic with a predefinedvalue, and issuing the control signal based on the comparison result.

The filter controller may include means for generating one of multiplevalues as the control signal to designate the pattern of filteringcoefficients of the filter.

The filter may be any one of a one dimensional filtering means, a twodimensional filtering means or a three dimensional filtering means whichincludes time axis as the third dimension.

The filter may be placed either before or after the frame memory.

In accordance with another aspect of the invention, an interframe codingsystem for frame-to-frame coding of an input signal, having a localdecoding loop which includes an encoder for interframe coding the inputsignal, decoding means for producing a decoded signal, and a framememory which receives the decoded signal and outputs a predictivesignal, may include a controller for receiving the input signal and thepredictive signal from the frame memory, for calculating a differencebetween the input signal and the predictive signal, and for generating acoding control signal for controlling the encoder.

The controller may include a difference calculator for receiving theinput signal and the predictive signal for calculating a differencebetween the input signal and the predictive signal and for outputting adifference signal, a pixel difference calculator for receiving thedifference signal, for calculating a value of the difference per eachpixel, and for outputting the difference value per each pixel to theencoder, and wherein the encoder may include means for quantizing theinput signal with a step size and for changing the step size ofquantization so that coding error becomes smaller than the differencevalue per each pixel.

The decoding loop may include a filter, and the controller may includemeans for generating a filter control signal to control the filterresponsive to the input signal and the predictive signal.

In accordance with yet another aspect of the invention, an interframecoding system for frame-to-frame coding of an input signal, having alocal decoding loop which includes an encoder for interframe coding theinput signal, decoding means for producing a decoded signal, and a framememory which receives the decoded signal and outputs a predictivesignal, may include a controller for receiving at least one of the inputsignal and the predictive signal, calculating an activity value from thereceived signals, and outputting an activity signal to the encoder tocontrol the coding therein.

In accordance with yet another aspect of the invention, an interframecoding method for frame-to-frame coding of an input signal, wherein acoding system includes a local decoding loop comprising a frame memoryfor receiving a decoded signal and outputting a predictive signal, afilter for receiving and filtering the predictive signal, may includethe steps of:

(a) calculating the difference between the input signal and thepredictive signal;

(b) decoding a filtering coefficient of the filter based on thedifference; and

(c) filtering the predictive signal based on the filtering coefficient.

In accordance with yet another aspect of the invention, an interframecoding method for frame-to-frame coding of an input signal, wherein acoding system includes a local decoding loop comprising an encoder forinterframe coding an input signal, decoding means for producing adecoded signal, a frame memory for receiving the decoded signal andoutputting a predictive signal, and a filter for receiving and filteringthe predictive signal, may include the steps of:

(a) calculating a characteristic of the input signal;

(b) deciding a filtering coefficient of the filter based on thecharacteristic; and

(c) filtering the signal based on the filtering coefficient.

In accordance with yet another aspect of the invention, an interframecoding method for frame-to-frame coding of an input signal, wherein thecoding system includes a local decoding lop including an encoder forinterframe coding an input signal, decoding means for producing adecoded signal, and a frame memory for receiving the decoded signal andoutputting a predictive signal, may include the steps of:

(a) calculating the difference between the input signal and thepredictive signal;

(b) changing a step size of quantization in the encoder responsive tothe difference; and

(c) coding the input signal based on the step size of quantization.

In accordance with yet another aspect of the invention, an interframecoding method for frame-to-frame coding of an input signal, wherein thecoding system includes a local decoding loop including an encoder forinterframe coding the input signal, decoding means for producing adecoded signal, and a frame memory for receiving the decoded signal andoutputting a predictive signal, may include the steps of:

(a) providing a characteristic signal indicating relative characteristicof one of the input signal and the predictive signal;

(b) changing a step size of quantization in the encoder based on thecharacteristic signal; and,

(c) coding the input signal based on the step size of quantization.

In accordance with yet another aspect of the invention, a coding systemwhich includes an encoder for coding an input signal into a coded signaland a frame memory for outputting a predictive signal, may include acoding controller responsive to the input signal and the predictivesignal, for generating a coding control signal for controlling theencoder.

The coding controller may include means for interframe coding whichproduces an interframe signal, means for lntraframe coding whichproduces an intraframe signal, means for determining characteristics ofthe interframe signal and the intraframe signal, means for comparing thetwo characteristics, and means for generating the coding control signalbased on the comparison result.

The comparing means may include means for weighting at least one ofcharacteristics.

The coding system may further include a filter for receiving andfiltering the predictive signal, and outputting a filtered signal, and afilter controller responsive to the input signal and the predictivesignal for generating a filter control signal to control the filter.

The coding controller may include means for interframe coding whichproduces a first interframe signal based on the filtered signal and asecond interframe signal based on the unfiltered predictive signal,means for intraframe coding which produces an intraframe signal, meansfor determining characteristics of the first and second interframesignals and the intraframe signal, means for comparing any two of thecharacteristic of the first interframe signal, the characteristic of thesecond interframe signal, and the characteristic of the intraframesignal, and means for generating the coding control signal based on thecomparison result.

The comparing means may include means for weighting at least one ofcharacteristics.

The coding controller may include means for interframe coding whichproduces an interframe signal based on the filtered signal, means forintraframe coding which produces an intraframe signal, means fordetermining a characteristic of the interframe signal and acharacteristic of the intraframe signal, means for comparing the twocharacteristics, and means for generating the coding control signalbased on the comparison result.

The comparing means may include means for weighting at least one ofcharacteristics.

The coding controller may include means for generating the codingcontrol signal based on the coded signal, forming a feed back loop.

The coding controller may include means for calculating a signalcharacteristic based on a difference between the maximum and minimumvalues of luminance intensity of pixels of the input signal, on the sumof the absolute difference values between the mean value of luminanceintensity of the pixels and the individual values, or on the sum of thesquared difference values between the mean value of luminance intensityof the pixels and the individual values.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram illustrating a configuration of aninterframe coding system according to an embodiment of the presentinvention;

FIG. 2 shows a block diagram of a configuration of a filter controllerin FIG. 1;

FIGS. 3(a), 3(b), 3(c) and 3(d) show explanation diagrams of a block ofpixels and equations of difference calculation according to theembodiment in reference to FIG. 1.

FIG. 4 is an explanation diagram of the "Activity" of an image signalaccording to the embodiment with reference to FIG. 1;

FIGS. 5(a), 5(b), 5(c) and 5(d) show diagrams of the "Activity" of animage signal according to the embodiment with reference to FIG. 1;

FIG. 6 shows a flowchart of filtering control operation according to theembodiment in reference to FIG. 1.

FIG. 7 shows a flowchart of another filtering control operationaccording to the embodiment.

FIG. 8 shows a flowchart of another filtering control operationaccording to the embodiment.

FIGS. 9(a), 9(b) and 9(c) are explanation diagrams of differentfiltering techniques of one-, two-, and three-dimensions according tothe embodiment with reference to FIG. 1;

FIG. 10 is an explanation diagram of intra- and inter-block filteringtechniques according to the embodiment with reference to FIG. 1;

FIGS. 11(a), 11(b), 11(c) and 11(d) are explanation diagrams of anoperation of an adaptive filter in FIG. 1;

FIG. 12 is an explanation diagram of filtering characteristics of animage signal according to the embodiment with reference to FIG. 1;

FIG. 13 shows a block diagram illustrating a configuration of aninterframe coding system according to another embodiment of the presentinvention;

FIG. 14 shows a block diagram illustrating a configuration of aninterframe coding system according to another embodiment of the presentinvention;

FIG. 15 shows a block diagram illustrating a configuration of aninterframe coding system according to another embodiment of the presentinvention;

FIG. 16 shows a block diagram illustrating a configuration of aninterframe coding system according to another embodiment of the presentinvention;

FIG. 17 shows a block diagram of a configuration of a filter controlleraccording to the embodiment with reference to FIG. 16;

FIGS. 18(a), 18(b), 18(c) and 18(d) are explanation diagrams of anoperation of an encoder according to the embodiment with reference toFIG. 16;

FIG. 19 shows a block diagram of a configuration of an interframe codingsystem according to another embodiment of the present invention;

FIGS. 20(a), 20(b) and 20(c) show block diagrams of a configuration of acontroller according to the embodiment with reference to FIG. 19;

FIGS. 21(a) and 20(b) are explanation diagrams of frame and fieldaccording to another embodiment of the present invention;

FIG. 22 shows a block diagram of a configuration of an interframe codingsystem according to another embodiment of the present invention;

FIG. 23 shows a block diagram of a configuration of a coding controlleraccording to the embodiment with reference to FIG. 22;

FIG. 24 shows a block diagram of a configuration of a "Activity"comparative selector according to the embodiment with reference to FIG.22;

FIG. 25 shows a block diagram of a configuration of an interframe codingsystem according to another embodiment of the present invention;

FIGS. 26(a) and 26(b) are explanation diagrams of weighting operated ina weighting circuit according to the embodiment with reference to FIG.25;

FIG. 27 shows a flowchart of the basic operation in the weightingcircuit according to the embodiment.

FIG. 28 shows a flowchart of an example of weighting operation in theweighting circuit according to the embodiment.

FIG. 29 shows a flowchart of the basic operation in the comparatoraccording to the embodiment.

FIG. 30 shows a flowchart of the basic operation in the control signalgenerator according to the embodiment.

FIGS. 31(a) and 31(b) show a memory table of the control signalgenerator according to the embodiment.

FIG. 32 shows a block diagram of a configuration of an interframe codingsystem according to another embodiment of the present invention;

FIG. 33 shows a block diagram of a configuration of an interframe codingsystem according to another embodiment of the present invention;

FIG. 34 shows a block diagram of a configuration of a coding controlleraccording to the embodiment.

FIG. 35 shows a block diagram of a configuration of an "Activity"comparative selector according to the embodiment.

FIG. 36 shows a block diagram of a configuration of an interframe codingsystem according to another embodiment of the present invention;

FIG. 37 shows a block diagram of a configuration of an interframe codingsystem according to another embodiment of the present invention;

FIG. 38 shows a block diagram of a configuration of an interframe codingsystem according to another embodiment of the present invention;

FIG. 39 shows a block diagram of a configuration of an interframe codingsystem according to another embodiment of the present invention;

FIG. 40 shows a block diagram of a conventional configuration of aninterframe coding system;

FIG. 41 shows basic characteristics of LPF filtering according to theconventional art;

FIGS. 42(a), 42(b) and 42(c) show a characteristic pattern of LPFfiltering according to the conventional art; and,

FIGS. 43(a), 43(b) and 43(c) show another characteristic pattern of LPFfiltering according to the conventional art.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiment 1.

FIG. 1 shows an interframe coding system in accordance with anembodiment of the present invention. FIG. 1 comprises FIG. 40 of theconventional art, as modified by adding a filter controller 21 and anadaptive filter 22 and by removing the filter 7 and the filtercontroller 8 instead. The interframe coding system processes an image bya processing unit or block of 8×8 pixels according to this embodiment.

Operation of the interframe coding system with a local decoding loopaccording to this embodiment is now described with reference to FIG. 1.

An input image signal 12 is processed in a motion vector detector 2 withan image signal 11 of the previous frame stored in the frame memory 1 todetect a motion vector. The frame memory generates a motion compensationpredictive signal 14 based upon a motion vector signal 13. Theinterframe coding process corresponds so far to that of the conventionalart. The motion compensation predictive signal 14 is input in the filtercontroller 21 together with the input image signal 12. The filtercontroller 21 generates a filter control signal 23.

Operation of the filter controller 21 according to this embodiment isnow described with reference to FIGS. 1 through 5.

FIG. 2 shows a configuration of the filter controller 21 in FIG. 1 whichincludes a difference calculator 30, a decision unit 31 and a differencesignal 32. The difference calculator 30 basically calculates adifference between the input image signal 12 and the motion compensationpredictive signal 14 to output the difference signal 32. Specifically,the difference signal 32 is output in the following process: thedifference calculator 30 calculates an absolute difference or squareddifference between the signals for each pixel Then the absolute valuesor squared values of pixel differences are calculated for the processingunit or a block of 8×8 (64) pixels and output as the difference signal32.

FIGS. 3(a), 3(b), 3(c) and 3(d) show explanation diagrams and equationsof difference calculation for the difference signal 32. FIGS. 3(a) and3(b) show a block or processing unit of 8×8 pixels of an image signal,respectively. FIG. 3(a) shows 64 pixels in a block of the input imagesignal 12 with each pixel represented by S1 through S64. FIG. 3(b) shows64 pixels in a block of the motion compensation predictive signal 14with each pixel represented by Y1 through Y64. FIGS. 3(c) and 3(d)represent possible forms of difference calculations of a luminanceintensity of the pixels between the input image signal 12 and the motioncompensation predictive signal 14 for the difference signal 32.Difference signal N in FIG. 3(c) is the sum of the absolute differencebetween the signals, while difference signal N in FIG. 3(d) is the sumof the squared difference between the signals.

The value of the difference signal 32 becomes smaller when thepredictive signal accurately predicts performance of the system. Thevalue becomes larger, on the other hand, for the following possiblecases:

(a) for a poor predictive signal resulting from a poor predictionperformance of the system;

(b) where the input image signal has varied significantly from theprevious image signal; and,

(c) for a poor predictive signal resulting from a poor quality of thelocal decoded signal held in the frame memory 1 containing a largeamount of quantization errors.

The decision unit 31 calculates an "Activity" of the block of pixels ofthe input image signal 12 responsive to the difference signal 32. The"Activity" is based upon the difference, for instance, between themaximum and minimum values of the input image signal 12 as shown in FIG.4.

FIG. 4 is an explanation diagram of the "Activity" illustrating that the"Activity" is based upon the difference between the maximum and minimumvalues of luminance intensity of pixels of the input image signal 12.The figure offers a brief explanation of the "Activity" by using onlyeight pixels instead of 64 pixels. The "Activity" of an image signalaccording to the figure is the difference value between the values ofluminance intensity of the fourth pixel (maximum intensity) and of thefifth pixel (minimum intensity) in the eight pixels of the image signal.

FIGS. 5(a), 5(b) and 5(c) show another example of calculating the"Activity" of the image signal which can be either the sum of theabsolute or squared difference values between the mean value ofluminance intensity of the pixels and the individual values.

FIG. 5(a) illustrates the "Activity" calculation of the mean value ofluminance intensity of the pixels of the image signal. X1 through X8 inthe diagram show the difference between the luminance intensity of thepixel and the mean value of luminance intensity of the eight pixels.FIG. 5(b) shows one of the calculation formulas of "Activity" X which isthe sum of the absolute difference values of X1 through X8. FIG. 5(c)shows another calculation formula of "Activity" X which is the sum ofthe squared difference values of X1 through X8.

The "Activity" denotes, as shown in FIGS. 4 and 5, nothing like anabsolute intensity but a relative value of an image signal.

Specifically, "Activity" X becomes smaller with an image signalcontaining a large amount of lower frequency components, "Activity" Xbecomes larger with an image signal containing a large amount of higherfrequency components.

The difference signal 32 is normalized or divided by the "Activity" ofthe image signal in the decision unit 31. A normalized result or valuedecides filtering intensity for eliminating higher frequency componentsin the image signal. The decision unit 31 generates the filter controlsignal 23 specifying filtering intensity according to a normalizedresult. Specifically, when a normalized value is larger, indicating thatthe image signal contains a great amount of higher frequency components,the image signal is filtered with higher filtering intensity. When anormalized value is smaller, on the other hand, indicating that thesignal contains a large amount of lower frequency components, the imagesignal is filtered with lower filtering intensity. The decision unit 31provides multiple values to the adaptive filter 22 for modifying thefiltering intensity of the filter. The filter control signal 23specifies filtering intensity of the image signal using one of themultiple values according to a normalized result.

The decision unit 31 according to this embodiment provides two values,for a brief explanation, indicating ON and OFF for controlling thefilter with the filter control signal 23. In this case, a thresholdvalue is to be set as a threshold for deciding whether the image signalis to be filtered or not. When a normalized value is larger than thethreshold, the signal is to be filtered for eliminating higher frequencycomponents and consequently reducing the amount of coding information.When a normalized value is smaller, on the other hand, than thethreshold, the image signal is unfiltered in order to keep its originaldefinition.

FIG. 6 shows a flowchart of filtering control in the filter controller21 controlling the filter operation (ON/OFF) with the motioncompensation predictive signal.

The filter controller 21 controls filtering based upon data from theinput image signal 12 and the motion compensation predictive signal 14.

The following is the operating sequences of filtering control with animage signal:

ST1. The adaptive filter is set OFF as an initial value.

ST2. The difference calculator 30 calculates the difference N betweenthe input image signal 12 and the motion compensation predictive signal14.

ST3. The decision unit 31 calculates the "Activity" of the input imagesignal 12.

ST6. The decision unit 31 normalizes or divides the difference N by the"Activity" X of the input image signal.

When a result of ST6 is smaller than a threshold value Th_(NX),indicating a successful prediction of an image signal, the motioncompensation predictive signal is not to be filtered and the filterremains OFF. This indicates that difference N is smaller or "Activity" Xis larger. In other words, the motion compensation predictive signalwith a smaller value of difference N has reproduced the imagesuccessfully and therefore does not need to be filtered. On the otherhand, the motion compensation predictive signal with a larger value of"Activity" X can indicate that the motion compensation predictive signal14 contains a large amount of higher frequency components due to theoriginal input image signal containing a large amount of higherfrequency components. The motion compensation predictive signal, in thiscase, is not to be filtered.

When a result of ST6 is larger than a threshold value Th_(NX),indicating a poor prediction of the image, the motion compensationpredictive signal is to be filtered and the filter is turned ON. Thisindicates that the difference N is larger or that the "Activity" X issmaller. In other words, the motion compensation predictive signal witha larger value of the difference N due to a poor prediction, thuscontaining a large amount of higher frequency components. Therefore themotion compensation predictive signal, in this case, should be filteredin order to eliminate the higher frequency components for a highlyefficient coding. On the other hand, the motion compensation predictivesignal with a smaller value of the "Activity" X is also to be filtered.However, there is a situation when the motion compensation predictivesignal with a larger value of "Activity" X is not to be filtered. It isoften the case when "Activity" X is larger, the motion compensationpredictive signal 14 contains a large amount of higher frequencycomponents. In this case the motion compensation predictive signal isnot to be filtered keeping the higher frequency components with it,which can achieve a higher coding efficiency.

FIG. 7 shows a flowchart of another filtering control methodaccomplished in the filter controller 21.

As ST4 in the figure shows, when difference N is larger than a firstthreshold value Th_(N), the filter is turned ON. This means that themotion compensation predictive signal contains a large amount of higherfrequency components and therefore is to be filtered in order toeliminate the higher frequency components to improve the codingefficiency.

As ST5 in the figure shows, when "Activity" X is smaller than a secondthreshold value Th_(X), the filter is turned ON. This indicates that theinput image signal 12 contains almost no higher frequency components. Inother words, the motion compensation predictive signal with less or nohigher frequency components cannot be influenced by filtering in termsof the elimination of higher frequency components and can still havemore effective result with filtering rather than without filtering interms of the elimination of noise. On the other hand, when "Activity" Xis larger than a threshold value Th_(X), the filter is not used (OFF).This indicates that the input image signal 12 contains a large amount ofhigher frequency components and, consequently, the motion compensationpredictive signal 14 also contains a large amount of higher frequencycomponents. In this case the motion compensation predictive signal 14remains unfiltered keeping higher frequency components with it.

FIG. 8 shows a flowchart of another filtering control method which canbe executed in the filter controller 21. FIG. 8 is a combination ofFIGS. 6 and 7. The combination brings about a more adaptive control offiltering the image signal with many choices of threshold valuesavailable.

With further reference to the embodiment, filtering control inaccordance with the invention can be operated based on the comparisonbetween the "Activity" of the input image signal and a threshold valueTh_(X). In other words, the flowchart in FIG. 7 can be altered byomitting the difference calculation step ST2 and the comparison step ST4between N and Th_(X).

When a division is operated by zero, the quotient is generally supposedto be infinite. This theory, however, does not apply to normalizationhere, for the calculation is generally with a finite length word. Inother words, when a difference results from the difference calculator 30is normalized by zero at ST6 in FIG. 6, a value of the "Activity", thequotient or normalized result is considered the maximum value of thefinite word used. Accordingly, when normalization uses an eight-bitoperation, the quotient is "FF", which is the maximum value of aneight-bit word, which is the maximum normalized value.

The adaptive filter 22 filters the motion compensation predictive signal14 according to the filter control signal 23 for eliminating higherfrequency components in the signal. As shown in FIG. 9, the adaptivefilter 22 can provide a multi-dimensional filtering including one- two-and three-dimensional filters. The three-dimensional filter can changethe filtering coefficient based upon the motion in an image signal asthe frame varies with time. Filtering intensity of the adaptive filtercan be varied according to the filtering coefficient specified by thefilter control signal 23.

FIGS. 9(a), 9(b) and 9(c) illustrate multi-dimensional low-passfiltering of the adaptive filter 22. FIG. 9(a) shows a one dimensionalfiltering of a line of pixels having a processing pixel as shown by ablack dot in the processing unit or block of pixels. FIG. 9(b) showshorizontal and vertical lines of pixels in the block, and illustrates atwo-dimensional filtering. FIG. 9(c) shows horizontal and vertical linesof pixels in blocks of time-varying frames t1, t2 and t3, therebyillustrating a three-dimensional filtering.

Black dots in FIG. 9 indicate processing pixels. A processing pixel canbe filtered with the influence of the neighboring pixels according toany of the one-, two-, and three-dimensional filtering techniques.

An image signal can be filtered either by, a intra-block filtering, orby using pixels within the processing unit or block of 8×8 pixelsexclusively, or by a inter-block filtering, or by using pixels in theneighboring blocks comprehensively as FIG. 10 shows.

FIG. 10 illustrates examples of the inter-block filtering with theprocessing pixel in a black dot. F1 and F2 in the figure show,respectively, the inter-block filtering of the processing pixel by usingpixels in the neighboring blocks.

FIGS. 11(a), 11(b), 11(c) and 11(d) are explanation diagramsillustrating a one-dimensional filtering.

FIG. 11(a) shows five pixels, S1 through S5, for a one-dimensionalfilter, including a black dot, S3, as the processing pixel. K1 throughK5 in the figure indicate filtering coefficients for each correspondingpixels S1 through S5 in the figure. The sum of the coefficients is 1.0as shown in FIG. 11(b).

Filtering intensity for the processing pixel is specified or decided bythe result of the following calculation: the luminance intensity of eachpixel of the processing pixel and the neighboring pixels S1 through S5is multiplied by a group of corresponding filtering coefficients k1through K5. Each result of the multiplication is summed up. FIGS. 11(c)and 11(d) show different patterns of filtering coefficients K1 throughK5 for the multiplication under the condition that the sum of thecoefficients is 1.0. When the value of filtering coefficient of theprocessing pixel is not 1.0 (K3=0.6) or those of the neighboring pixelsare non-zero (K1=K2=K4=K5=0.1) as shown in FIG. 11(c), the luminanceintensity of the processing pixel is influenced by those of theneighboring pixels to be normalized. When the value of filteringcoefficient of the processing pixel is 1.0 (K3=1.0) or those of theneighboring pixels are zero (K1,K2,K4,K5=0.0) as shown in FIG. 11(d),the luminance intensity of the processing pixel has no influence bythose of the neighboring pixels keeping its original luminanceintensity. This indicates that the processing pixel is not to befiltered or the low-pass filter is to be OFF.

FIG. 12 is an explanation diagram illustrating different characteristicsof the predictive signal 24 by different degrees of filtering intensitywith different filtering coefficients. The filter control signal 23specifies the optimal filtering intensity for the motion compensationpredictive signal 14 to eliminate higher frequency components in thesignal in a multiphase manner with multiple choices of the pattern offiltering coefficients. The four characteristic of the predictionsignal, 24a through 24b, in the figure lndlcate different filteringresults by multiphase elimination of higher frequency components in thesignal with different patterns of filtering coefficients. Characteristic24a shows that the motion compensation predictive signal 14 is notfiltered containing a great amount of higher frequency components.Characteristic 24d shows that the motion compensation predictive signal14 is filtered containing a large amount of lower frequency componentswith very small portion of or no higher frequency components. Thecharacteristics of the signal can be changed with different filteringcoefficients from 24a to 24d as an arrow A indicates if the predictivesignal contains a large amount of higher frequency components. As anarrow B indicates, the characteristic can be changed from 24d to 24awith different filtering coefficient if the signal requlres higherdefinition. Generally, an image signal can keep its original resolution,its highest definition, wlthout filtering.

When filtered the predictive signal 24 is subtracted from the inputimage signal 12 in a subtractor 3, where a predictive error signal 15 isoutput. The predictive error signal 15 is coded by quantization in anencoder 4, where a coded error signal 16 is output. The coded errorsignal 16 is decoded in a local decoder 5, where a local decoded errorsignal 17 is output. The local decoded error signal 17 is added to thepredictive signal 24 in an adder 6, where a local decoded signal 18 isoutput. The local decoded signal is stored or written in the framememory 1.

Embodiment 2.

With further reference to FIG. 1, the interframe coding system inaccordance with the invention can omit the motion vector detector 2 interms of simplification of the system as shown in FIG. 13. In this case,the quantity of motion of an image signal is to be considered zero.

Embodiment 3.

With further reference to FIG. 1, the interframe coding system inaccordance with the invention can provide the adaptive filter 22 afterthe adder 6 in the local decoding loop instead of being after the framememory 1, as is shown in FIG. 14.

Embodiment 4.

With further reference to FIG. 1, the interframe coding system inaccordance with the invention can combine the previous two embodiments.The interframe coding system omits the motion vector detector 2 andprovides the adaptive filter 22 after the adder 6 in the local decodingloop instead of being after the frame memory 1, as is shown in FIG. 15.In FIG. 15 the motion vector detector 2 is not provided. Therefore, theinterframe coding system with this configuration operates under thecondition that the quantity of motion of an image signal is zero.

Embodiment 5.

FIG. 16 shows another configuration of the interframe coding system inaccordance with the invention. The filter controller 21 outputs a pixeldifference signal 25 or a control signal to the encoder 4 as well as thefilter control signal 23 to the adaptive filter 22. The pixel differencesignal 25 is the difference per pixel which is calculated by dividingthe value of the difference signal 32 by the number of pixels in a blockof pixels (e.g. 8×8=64) in the filter controller 21.

FIG. 17 shows a configuration of the filter controller 21 according tothis embodiment. FIG. 17 comprises FIG. 2, as modified by adding a pixeldifference calculator 33 for calculating the pixel difference signal 25.The difference calculator 30 outputs the difference signal 32 by thefollowing way stated hereinbefore in reference to FIG. 2. The absoluteor squared difference values of luminance intensity of pixels betweenthe input image signal 12 and the motion compensation predictive signal14 are summated within a block of pixels (8×8=64).

The value of the difference signal 32 is divided in the pixel differencecalculator 33 by the number of pixels in a block of pixels (8×8=64) toproduce the average value of the difference signal 32 in order to outputthe pixel difference signal 25. The pixel difference signal 25 is thethreshold value of quantization which is output to the encoder 4. Theencoder 4 decides the optimal quantization step size for coding theimage signal under the condition that quantization error does not exceedthe value of the pixel difference signal 25.

FIGS. 18(a), 18(b), 18(c) and 18(d) show the pixel difference of animage signal illustrating the operation of deciding the quantizationstep size in the encoder 4 in accordance with this embodiment.

In FIG. 18(a), D1 designates the pixel difference of a block containingeight pixels, D2 designates the pixel difference of another block ofeight pixels, and D3 designates the pixel difference of another block ofeight pixels. Those pixel differences are supposed to be given thefollowing condition in the filter controller 21 as shown in FIG. 18(b):D1>D2>D3.

When the pixel difference signal 25 is input into the encoder 4, theencoder 4 selects the optimal quantization step size for an input imagesignal based upon the pixel difference signal 25, so that the value ofquantization error should be less than the value of the pixel differencesignal 25. With D1 as the pixel difference signal 25, for instance, theencoder 4 selects a quantization step size SS1 (SS1<D1), with D2 as thesignal a step size SS2 (SS2<D2) is selected; and a step size SS3(SS3<D3) is selected for D3.

Under that condition, the encoder 4 quantizes an input image signal byselecting the optimal quantization step sizes based upon the value onthe pixel difference signal 25 in the following relations as shown inFIG. 18(d): SS1>SS2>SS3.

Thus, the quantization step size becomes larger when the differencebetween the input image signal 12 and the motion compensation predictivesignal 14 is large. On the other hand, the quantization step sizebecomes smaller when the difference between the two signals is smaller.Consequently this leads to a highly efficient coding performanceminimizing the quantization error to have a high quality decoded image.

The pixel difference signal 25 is not the only signal used to decidedthe quantization step size in the encoder 4. The quantization step sizeis also controlled or limited by the size of an output buffer whichtemporarily stores coded signals to be output from the encoder 4. Thesize of the buffer can limit the quantity of coding signals. In thiscase, the quantization step size is limited by the size of the outputbuffer.

Embodiment 6.

With further reference to FIG. 16, the interframe coding system inaccordance with the invention can provide a controller 21a to output acontrol signal 25 to the encoder 4 for controlling the quantization stepsize. In this embodiment the adaptive filter 22 and the filter controlsignal 23 are not necessarily provided as shown in FIG. 19.

FIG. 20(a) shows a configuration of the controller 21a according to thisembodiment. FIG. 20(a) comprises FIG. 17, as modified by removing thedecision unit 31. The operation of the difference calculator 30 and thepixel difference calculator 33 corresponds to that described inreference to FIG. 17.

FIG. 20(b) shows another configuration of the controller 21a accordingto this embodiment providing a decision unit 31a which outputs thecontrol signal 25 of the "Activity" of an image signal. The "Activity"is calculated with the input image signal 12 as shown in FIGS. 4 and 5.

The "Activity" of an image signal has a relative value, and not theabsolute value, of luminance intensity of an image signal as statedhereinbefore. In other words, "Activity" X in FIG. 5 becomes smallerwith an image signal containing a great amount of lower frequencycomponents, while it becomes larger with an image signal containing agreat amount of higher frequency components.

Accordingly, the control signal of the "Activity" enables the encoder 4to perform the optimal coding by selecting flexibly the optimalquantization step size for an image signal.

With further reference to FIG. 20(b), the decision unit 31b inaccordance with the invention can have the motion compensationprediction signal 14 as input information for calculating its "Activity"as shown in FIG. 20(c). The overall operation of the decision unit 31bis equivalent with that described in FIG. 20(b).

Embodiment 7.

With further reference to the embodiments hereinbefore, the frame memory1 in accordance with the invention can store an image signal not by theframe but by the field as shown in FIG. 21. FIG. 21 shows the relationbetween the frame and field of an image signal. FIG. 21(a) illustratesthat the frame of an image signal is consisted of two fields of an imagesignal, the first and second fields. FIG. 21(b) illustrates a framecomposed of the first and second fields of an image signal by using aninterlace mode.

With further reference to this embodiment, the filtering of an imagesignal can also be processed by the field.

With further reference to the embodiments hereinbefore, the frame memory1 in accordance with the invention can store a multiple number oftime-varying frames or fields including the past, present, andforthcoming frames or fields.

Embodiment 8.

With further reference to the embodiments hereinbefore with FIGS. 2 and17, the decision unit 31 of the invention can use the motioncompensation predictive signal 14, instead of the input image signal 12,to calculate its "Activity" for normalizing the difference signal 32.

Embodiment 9.

FIG. 22 shows a configuration of interframe coding system according toanother embodiment of the present invention. FIG. 22 comprises FIG. 1,as modified by adding a coding controller 41, a coding control signal 42and a select signal 150, and by removing the adaptive filter 22, thefilter controller 21 and the motion vector detector 2.

Operation with the local decoding loop of the interframe coding systemaccording to this embodiment is now described with reference to FIG. 22.The image signal 11 of the previous frame stored in the frame memory 1or the local decoded Image signal is used as a predictive image signal.The subtractor 3 subtracts the input image signal 12 from the predictiveimage signal 11 to output the predictive error signal 15. The codingcontroller 41 calculates the "Activities" of the input image signal 12and the predictive error signal 15 to output the coding control signal42 and the select signal 150, respectively, to the encoder 4.

The encoder quantizes the select signal 150 to output the coded errorsignal 16. The local decoder 5 decodes the coded error signal 18 tooutput the local decoded error signal 17. The adder 8 combines the localdecoded error signal 17 with the predictive image signal 11 to outputthe local decoded signal 18. The local decoded signal 18 is stored inthe frame memory 1. The coded error signal 16 is, on the other hand,also transmitted through a transmission line.

Operation of the coding controller 41 in accordance with this embodimentis now described with reference to FIG. 23. The coding controller 41 hasan "Activity" calculator 45, an "Activity" comparative selector 46, an"Activity" signal 12a based upon the input image signal 12, an"Activity" signal 15a based upon the predictive error signal 15, thecoding control signal 42 and the select signal 150.

In the coding controller 41, the "Activity" calculator 45 calculateseach "Activity" of the input image signal 12 and the predictive errorsignal 15 to output their "Activities" to the "Activity" comparativeselector 46.

The "Activity" of an image signal can be the difference between themaximum and minimum values of luminance intensity of pixels of a signalor the sum of the absolute or squared values of the difference betweenthe mean value and the values of luminance intensity of a block ofpixels in an Image signal as stated hereinbefore with reference to FIGS.4 and 5. The "Activity" does not refer to the absolute luminanceintensity of each pixel in an image signal. It involves a block ofpixels of an image signal. When a block of pixels of an image signalcontain a great amount of lower frequency components its "Activity"becomes smaller, while when they contain a great amount of higherfrequency components its "Activity" becomes larger.

In the "Activity" comparative selector 46, the "Activities" 12a and 15aare compared to select an optimal mode signal under a given prioritybased upon the user's or system's needs. The priority could be:

(1) to prioritize coding efficiency;

(2) to prioritize picture quality; or,

(3) to put significance both on the coding efficiency and the picturequality and prioritize either of them if one has more significancedepending on a given situation.

There are more possible significant items can be added to the list ofcondition stated above, apart from coding efficiency and picturequality. Condition can be limited depending on the purpose or capacityof the system or the user's needs. When coding efficiency isprioritized, for instance, a mode signal of the image signal withsmaller "Activity" by the comparison is output by the "Activity"comparison selector 46 as stated hereinbefore. On the other hand, whenpicture quality is prioritized, the mode signal of the input imagesignal 12 is output by the "Activity" comparison selector 46. When bothof coding efficiency and picture quality are prioritized, the "Activity"comparative selector 46 outputs the mode signal of either the inputimage signal 12 or the predictive error signal 15 based upon thecomparison result of their "Activities" under a certain consideration.

The "Activity" comparative selector 46 outputs the coding control signal42 providing coding parameters. The coding parameters include thequantization step size based upon the condition and the "Activities" ofthe image signals. The "Activity" comparison selector 46 also outputsthe image signal selected by the mode signal as the select signal 150.

Operation of the "Activity" comparative selector 46 according to thisembodiment is now described with reference to FIG. 24. The "Activity"comparative selector 46 has a weighting circuit 46a, a comparator 46band a control signal generator 46c. The weighting circuit 46a weightsthe "Activity" of the input signal 12a and/or the "Activity" of thepredictive error signal 15a. The comparator 46b compares the "Activity"of the input image signal 12a with the "Activity" of the predictiveerror signal 15a output from the weighting circuit 46a. The controlsignal generator 46c selects either the input image signal 12 or thepredictive error signal 15 as the select signal 150 and also outputs thecoding control signal 42 by multiplexing the quantization step size,coding coefficients, etc. The weighting circuit 46a will be left to thelater discussion for a brief description of the "Activity" comparativeselector 46 in this embodiment. Accordingly, the comparator 46b inputsthe "Activity" of the input image signal 12a and the "Activity" of thepredictive error signal 15a. The comparator 46b compares those"Activities" and outputs a mode signal 46m based upon the comparisonresult to the control signal generator 46c. The mode signal 46m actuatesa switch to input either the input image signal 12 or the predictiveerror signal 15 in the control signal generator 46c. The mode signal 46mis output as a part of the coding control signal 42 as one of the codingparameters included in the multiplexed signal together with thequantization step size, discrete cosine transform (DCT) coefficients,etc. to the encoder 4. The encoder 4 demultiplexes and checks themultiplexed coding control signal 42, whereby the encoder 4 is informed,for instance with the mode signal, which signal is selected as theselect signal 150 of the input image signal 12 and the predictive errorsignal 15. When the input image signal 12 is selected as the selectsignal 150, the encoder 4 performs "intraframe" coding, while thepredictive error signal 15 is selected, the encoder 4 performs"interframe" coding. Interframe and intraframe codings are controlled byquantization step sizes and the DCT coefficients.

Embodiment 10.

With further reference to the previous embodiment referring to FIG. 22,the interframe coding system in accordance with time invention can havea filter controller, an adaptive filter and a motion vector detector asshown in FIG. 25. Accordingly, FIG. 25 comprises FIG. 22, as modified byadding the motion vector detector 2, the motion vector signal 13, themotion compensation predictive signal 14, the filter controller 21, theadaptive filter 22, the filter control signal 23 and the predictivesignal 24.

Operations of the motion vector detector 2, the filter controller 21 andthe adaptive filter 22 correspond to those described in the firstembodiment referring to FIG. 1.

With further reference to FIG. 22, the predictive error signal 15 inaccordance with the invention can be based partly upon the filteredpredictive signal 24 through the filter controller 21 and the adaptivefilter 22 as shown in FIG. 25 instead of the predictive signal 11unfiltered in FIG. 22. In other words, the predictive error signal 15 isthe subtracted result of the input image signal 12 by the filteredpredictive signal 24 in the subtractor 3. Consequently, the codingcontroller 41 according to this embodiment compares the "Activity" ofthe input image signal 12 and the "Activity" of the filtered predictiveerror signal 15 to output the coding control signal 42.

The filter controller 21 according to this embodiment calculates thedifference between the input image signal 12 and the Image signal fromthe frame memory 1 or the motion compensation predictive signal 14. Thefilter controller 21 normalizes the difference by the "Activity" ofeither the input image signal 12 or the motion compensation predictivesignal 14. Accordingly, the optimal filtering intensity for the motioncompensation predictive signal 14 is decided based upon the normalizedresult. This can contribute to a desirable elimination of higherfrequency components in the image signal leaving no higher frequencycomponents in the prediction error signal, which leads to a high codingefficiency. The coding control signal 42 according to this embodimentcan be output by the coding controller 41 based upon the mode signal.The mode signal is to select the image signal having smaller "Activity"of the two: the "Activities" of the input image signal 12a and the"Activity" of the optimally filtered prediction error signal 15a. Thisalso contributes to a high coding efficiency.

Embodiment 11.

With further reference to the previous two embodiments, the "Activity"comparative selector 46 in the coding controller 41 in accordance withthe invention can select the mode signal 46m based only upon the inputsignal resulted in the smallest "Activity" in the "Activity" comparison.Accordingly, the "Activity" comparative selector 46 outputs the modesignal based upon the smallest "Activity" in the comparator and outputsthe coding control signal 42 of coding parameter, including thequantization step size, etc. based upon the mode signal and the"Activity".

An image signal with smaller "Activity" indicates that the image signalcan be coded with higher coding efficiency in the encoder.

Embodiment 12.

With further reference to the previous two embodiments or FIGS. 22 and25, the "Activity" comparative selector 46 in accordance with theinvention can weight "Activities" of the input signals, at least one ofthem, for comparison.

The significance and operation of weighting "Activity" operated in theweighting circuit 46a in the "Activity" comparative selector 46 in FIG.24 is now described with reference to FIGS. 24 and 26. FIGS. 26(a) and26(b) show the relation between the "Activities" of input signals andweighting. FIG. 26(a) shows the original "Activity" of the input imagesignal 12a and the original "Activity" of the predictive error signalwith no weight added in the weighting circuit 46a. FIG. 26(b) shows theoriginal "Activity" of the input image signal 12a with no weight addedand the "Activity" of the predictive error signal with twice as muchweight added as the original "Activity".

Under the condition that the "Activity" comparative selector 46 is toselect the mode of an input signal with the smallest "Activity", the"Activity" comparative selector 46 selects the predictive error signal15 as the select signal 150 according to FIG. 26(a). The figure showsthat the "Activity" of the predictive error signal 15a is constantlysmaller than the "Activity" of the input image signal 12a.

Under that condition, the "Activity" comparative selector 46, on theother hand, selects either of the signals alternatively of the inputimage signal 12 or the predictive error signal 15 added a twofold weighton its "Activity" 15b when it has smaller "Activity" than the other asthe select signal 150 according to FIG. 26(b). The figure shows that thepredictive error signal 15 is selected during the time period T0 throughT1 as the select signal 150. The input image signal 12 is selectedduring the time period T1 through T2 as the select signal 150 due to theweighting on the predictive error signal.

FIG. 27 shows a flowchart of the basic operation in the weightingcircuit 46a according to the embodiment in reference to FIG. 24. Whenthe weighting circuit 46a inputs the "Activity" of the input imagesignal 12a and the "Activity" of the predictive error signal (differencesignal between the input image signal 12 and the predictive signal fromthe frame memory) 15a, the circuit reads out the previously assignedweighting coefficients W1 and W2, respectively, for the signals. Theinput "Activities" of the signals 12a and 15a are multiplied by theweighting coefficients W1 and W2, respectively, and a weighted"Activity" of the input image signal 12b and a weighted "Activity" ofthe predictive error signal 15b is output.

With further reference to this embodiment, the weighting circuit 46a inaccordance with the invention can weight only one of the "Activities" ofthe input signals instead of both of them.

With further reference to the embodiment, the weighting circuit 46a inaccordance with the invention can process three or more input signals,instead of two as described in this embodiment. An example of usingthree input signals in that condition will be described later inreference to FIG. 33.

FIG. 28 shows a flowchart of an example of weighting operation in theweighting circuit 46a according to the embodiment in reference to FIG.26. In this example, the "Activity" of the input image signal 12a is notweighted and output as "Activity" 12b after the weighting circuit. The"Activity" of the predictive error signal 15a is doubled by the weightand output as its weighted "Activity" 15b.

FIG. 29 shows a flowchart of the basic operation in the comparator 46baccording to the embodiment in reference to FIG. 24.

The comparator 46b compares the input "Activities" of the input imagesignal 12b and of the predictive error signal 15b from the weightingcircuit 46a to decide a coding mode. When "Activity" 12b is smaller than"Activity" 15b, the comparator 46b selects INTRA mode (for an intraframecoding) while the comparator selects INTER mode (for an interframecoding) when "Activity" 12b is larger than "Activity" 15b. When thecomparator selects the mode, it outputs mode signal 46m to the controlsignal generator 46c.

With further reference to the embodiment, the weighting circuit 46a inaccordance with the invention can process three or more input signals,instead of two as described in this embodiment. An example of usingthree input signals in that situation will be described later inreference to FIG. 33.

FIG. 30 shows a flowchart of the basic operation in the control signalgenerator 46c according to the embodiment in reference to FIG. 24.

The control signal generator 46c inputs the input image signal 12, thepredictive error signal 15, the weighted "Activity" of the input imagesignal 12b, the weighted "Activity" of the predictive error signal 15band the mode signal 46m. The control signal generator 46c outputs thecontrol signal 42 and one of the input signals as the select signal 150based upon the mode signal 46m. The control signal generator 46c isinformed of the selection of either the INTER or INTRA mode with themode signal 46m. When the INTER mode is selected, the control signalgenerator outputs the predictive error signal 15 as the select signal150. When the INTRA mode is selected, the control signal generatoroutputs the input image signal 12 as the select signal 150. The controlsignal generator 46c reads out the optimal quantization step size(QUANT) and the number of the DCT coefficients in the memory table inFIG. 31 according to the weighted "Activity" of the selected signal 12bor 15b. The readout quantization step size (QUANT) and DCT coefficientsas well as the mode signal 46m are multiplexed to be output as thecontrol signal 42.

With further reference to the embodiment, the control signal generator46c in accordance with the invention can input the "Activity" of theinput image signal 12a and the "Activity" of the predictive error signal15a, instead of the weighted "Activities" of those signals 12b and 15b.In this case, the threshold values or the contents of the memory tablein FIG. 31 have to be altered.

With further reference to the embodiment, the number of threshold valuein FIG. 31 in accordance with the invention can be varied according tothe assigned QUANT and DCT coefficients. The values of the QUANT and DCTcoefficients can be also assigned based upon a given condition orpriority, such as the picture quality or coding efficiency.

Thus, selective, intentional control of an output signal, or the selectsignal 150 can be realized by the weighting technique. In other words,weighting can control the selection probability of input image signalsin the "Activity" comparative selector 46. The select signal 150 can becontrolled by weighting technique which is provided with previously setweighting coefficients in the weighting circuit according to the user'sor system's needs, or according to the picture quality desired.

Embodiment 13.

With further reference to FIG. 25, the coding controller 41 inaccordance with the invention can compare the "Activity" of the filteredpredictive error signal with the "Activity" of the unfiltered predictiveerror signal to output the coding control signal 24 based upon thecompared result, specifically, the signal with smaller "Activity".

FIG. 32 shows a configuration of interframe coding system according tothis embodiment providing a predictive signal 24a filtered in theadaptive filter 22 and a predictive signal 24b unfiltered or the motioncompensation predictive signal 14. Those predictive error signals 24aand 24b are subtracted, respectively, from the input image signal 12 inthe subtractors 3a and 3b. The differences resulted from the subtractionare output as the predictive error signals respectively. The predictiveerror signals are compared in the coding controller 41 to output thecoding control signal 42 in the same manner described hereinbefore.

Embodiment 14.

With further reference to FIG. 32 or the previous embodiment, the codingcontroller 41 in accordance with the invention can add weight on eitheror both of the "Activities" of the predictive signals, filtered in theadaptive filter 22 or/and unfiltered, for comparison to output thecoding control signal 42.

Embodiment 15.

With further reference to FIG. 32, the coding control signal 42 inaccordance with the invention can be based upon the compared resultamong the "Activities" of three signals, the filtered predictive errorsignal and the unfiltered predictive error signal, and the input imagesignal 12, specifically selecting the mode of the signal with thesmallest "Activity"

FIG. 33 shows a configuration of interframe coding system according tothis embodiment. FIG. 33 comprises FIG. 32, as modified by adding theinput image signal 12 input in the coding controller 41. The codingcontroller 41 uses three input signals of the input image signal 12, thefiltered predictive error signal 15x and the unfiltered predictive errorsignal 15y for the "Activity" comparison to output the coding controlsignal 42 by selecting the mode of the signal with the smallest"Activity".

FIG. 34 shows a block diagram of a configuration of the codingcontroller 41 according to the embodiment. FIG. 34 comprises FIG. 23, asmodified by adding a filtered predictive error signal 15x and anunfiltered predictive error signal 15y, instead of the predictive errorsignal 15 and also adding an "Activity" of the filtered predictive errorsignal 15a and an "Activity" of the unfiltered predictive error signal15c. Therefore the coding controller 41 according to this embodimentinputs three signals, instead of two as described hereinbefore. The"Activity" calculator 45 calculates the "Activities" of the threesignals and outputs the calculated result to the "Activity" comparativeselector 46. FIG. 35 shows a block diagram of a configuration of the"Activity" comparative selector 46 according to the embodiment. FIG. 35comprises FIG. 24, as modified by adding the filtered predictive errorsignal 15x, the unfiltered predictive error signal 15y, the "Activity"of the predictive error signal 15c, and a weighted "Activity" of thepredictive error signal 15d. The three input "Activities" of the signalsin the "Activity" comparative selector 46 are added weight in theweighting circuit 46a. Weighting coefficients W1, W2, and W3 is assignedin the weighting circuit in advance. The weighting circuit adds weightby multiplying the "Activity" of input image signal 12a by a weightingcoefficient W1 to output a weighted "Activity" of the input image signal12b to the comparator 46b. In the same manner, the "Activity" 15a ismultiplied by W2 to have a weighted "Activity" 15b and the "Activity"15c is multiplied by W3 to have a weighted "Activity" 15d. Thecomparator 46b compares the three weighted "Activities" to generate amode signal 46m based upon the comparison result. The control signalgenerator 46c selects one of the three signals as the output selectsignal 150, of the input image signal 12, the filtered predictive errorsignal 15x, and the unfiltered predictive error signal 15y based uponthe mode signal 46m. When the input image signal 12 is selected for theselect signal 150, the control signal generator 46c reads out aquantization step size and a DCT coefficient previously stored in thememory table according to the value of the weighted "Activity" of theinput image signal 12b. When the filtered predictive error signal 15x isselected for the select signal 150, the control signal generator 46creads out a quantization step size and a DCT coefficient in that manneras described hereinbefore based upon the value of the weighted"Activity" of the filtered predictive error signal 15b. When theunfiltered predictive error signal 15y is selected for the select signal150, the control signal generator 46c reads out a quantization step sizeand a DCT coefficient in that manner as described hereinbefore basedupon the value of the weighted "Activity" of the unfiltered predictiveerror signal 15d. The readout quantization step size and DCT coefficientas well as the mode signal 46m are multiplexed to be output as thecontrol signal 42.

Embodiment 16.

With further reference to FIG. 33 or the previous embodiment, the codingcontroller 41 in accordance with the invention can add weight on atleast one of the "Activities" of the three signals, the filteredpredictive signal 15a, the unfiltered predictive error signal 15, andthe input image signal 12 for the "Activity" comparison to output thecoding control signal 42 based upon the signal with the smallest"Activity".

Embodiment 17.

With further reference to the previous embodiment, the coding controlsignal 42 in accordance with the invention can be output based upon amode signal derived from an image signal with smaller "Activity"compared between those of the filtered predictive error signal 15a andthe input image signal 12a.

Embodiment 18.

With further reference to the previous embodiment, the coding controller41 in accordance with the invention can add weight on either or both ofthe "Activities" of the filtered predictive error signal through theprocessing units or/and the input image signal 12 for comparison tooutput the coding control signal 42.

FIG. 36 shows a configuration of interframe coding system according tothis embodiment. FIG. 36 comprises FIG. 25, as modified by addinganother input of the filter control signal 23 from the filter controller21 into the coding controller 41. The filter control signal 23 is for aweighting control. The filter control signal informs the codingcontroller 41 of the condition of the predictive error signal 15 whetherit is filtered or not in the adaptive filter 22. The control signal 23is analyzed in the weighting circuit 46 in the coding controller 41 toinform whether the adaptive filter 22 is ON or OFF. When the weightingcircuit 46 knows that the predictive error signal 15 is based upon thepredictive signal 24 filtered in the adaptive filter 22, for example, itadds weight on the predictive error signal 15. When the weightingcircuit 46 knows that the predictive error signal 15 is based upon thepredictive error signal 24 unfiltered in the adaptive filter 22, on theother hand, it does not add weight on the predictive error signal.Dynamic weighting control can be realized with a variety of modes orvalues in the filter control signal 23 providing more than two modes ofON and OFF.

Embodiment 19.

With further reference to FIG. 25, the filter control signal 23 inaccordance with the invention can be output the filter control signal 23based just upon the difference between the input image signal 12 and theimage signal from the frame memory 1 without normalizing the differencein the filter controller 21.

Embodiment 20.

With further reference to FIG. 25, the filter control signal 23 inaccordance with the invention can be output based upon informationrelated to the difference between the input image signal 12 and theimage signal from the frame memory 1.

Embodiment 21.

With further reference to FIG. 25, the filter controller 21 inaccordance with the invention can output the filter control signal 23 bynormalizing the difference between the input image signal 12 and animage signal from the frame memory 1 by the image signal from the framememory 1.

Embodiment 22.

With further reference to FIG. 25, the adaptive filter 22 in accordancewith the invention can be provided after the adder 6 in the localdecoding loop as shown in FIG. 37. The adaptive filter 22 filters thelocal decoded signal 18 based upon the filter control signal 23 from thefilter controller 21. As a result, either the filtered or unfilteredlocal decoded signal is stored in the frame memory 1. The filtered, forinstance, image signal 14 or predictive signal 24 from the frame memory1 is subtracted from the input image signal 12 in the subtractor 3 tooutput the predictive error signal 15 for interframe coding.

Embodiment 23.

The frame memory 1 in accordance with the invention can store an imagesignal by the field instead of the frame. The frame memory can alsostore a multiple number of time-varying frames or fields instead oneframe or one field.

Embodiment 24.

FIG. 38 shows a configuration of interframe coding system in accordancewith the invention. FIG. 38 comprises FIG. 25, as modified by adding acoding-filtering controller 50 instead of providing separately thefilter controller 21 and the coding controller 41. Both of the filtercontroller and coding controller calculate the "Activity" of an imagesignal in the same manner to control a signal and therefore it may bedesirable to integrate them in terms of the concentration of the similarunit including the same function. Thus, the integrated controller offiltering and coding can provide more control efficient coding systemand contribute to downsizing or minimization of the overall system.

Embodiment 25.

With further reference to FIG. 25, the motion vector detector 2 inaccordance with the invention can be removed from the system when thesystem prioritizes simplification. With this configuration, the quantityof motion of an image signal is always considered zero.

Embodiment 26.

With further reference to FIGS. 22 through 38, coding control system inaccordance with the invention can provide a feedback loop based upon thecoded data from the encoder 4 or the local error signal 16, adding tothe coding controller 41 or the filtering-coding controller 50.

FIG. 39 shows a configuration of the interframe coding system accordingto this embodiment. FIG. 39 comprises FIG. 22, as modified by adding atransmission buffer 100, a transmission signal 160 and a feedback signal101. The transmission buffer 100 stores coded data from the encoder 4based upon the coded error signal 16. The transmission signal 160 of thecoded error signal 16 is transmitted onto a transmission line from thetransmission buffer 100. The feedback signal 101 is fed back to thecoding controller 41 from the transmission buffer 100.

Accordingly, the coding controller 41 controls coding with the codingcontrol signal 42 by the feedback signal 101. The feedback signal 101indicates, for instance, occupied ratio of coded error signals 16 in thetransmission buffer 100. In reception of the occupied ratio, the codingcontroller 41 controls coding with the coding control signal 42 toreduce the amount coding in the encoder 4, for instance, when the codederror signal 16 occupies the transmission buffer 100 in largerproportion. On the contrary, the coding controller 41 controls coding toincrease the amount of coding in the encoder 4 with the coding controlsignal 42, for instance, when the coded error signal 16 occupies thetransmission buffer 100 in smaller proportion.

Embodiment 27.

With further reference to the foregoing embodiments, the "Activity" ofan image signal in accordance with the invention can be represented in avariety of manners as a characteristic of the image signal. Theforegoing embodiments provide the "Activity", a characteristic, of animage signal by a difference between image signals or by the sum of theabsolute or squared difference values of an image signal vs. the meanvalue. There are other possible forms of the "Activity" of an imagesignal using other difference signals or the variance technique.

Embodiment 28.

With further reference to the foregoing embodiments, the processing unitof image signal in accordance with the invention can be a block of16×16, 32×32, or 8×16 pixels for example, instead of a block of 8×8pixels. Otherwise, the processing unit can possibly be other forms,instead of a block of pixels.

Embodiment 29.

With further reference to the foregoing embodiments, a coding signal orcoding data in accordance with the invention can be based upon radarsignal or sound signal, instead of image signal.

As described hereinbefore, one of the distinctive features of theinterframe coding system of the present invention is that the filtercontroller normalizes a difference between the input image signal andthe signal from the frame memory by either the input image signal or thesignal from the frame memory. The filter controller generates the filtercontrol signal based upon the normalization to control filteringintensity for eliminating higher frequency components in the signal. Thefilter controller also generates the filter control signal based uponcomparison between the "Activity" of the input image signal or ofdifference and a threshold value. Accordingly, filtering can be operatedadaptively to each image signal, so that an image signal can be providedfree from higher frequency components in the encoder for coding. Thiscan contribute to a highly efficient coding leading to a high definitioncoded image even with motion.

Another distinctive feature of the interframe coding system of thepresent invention is that the coding controller controls thequantization step size based upon one of the three signals which are adifference signal between the input image signal and the signal from theframe memory, the input image signal, and the signal from the framememory. This can contribute to reducing the amount of coding error in acoded image signal.

Another distinctive feature of the present invention is that the codingcontroller controls coding in the encoder by the mode signal based uponthe signal with the optimal "Activity" or characteristic of either theinput image signal or the predictive error signal. This can contributeto a highly efficient coding.

As stated hereinbefore, the interframe coding system and method of thepresent Invention have the following advantageous effects.

One of the advantageous features is that the filter controller controlsfiltering intensity for eliminating higher frequency components in animage signal by using the difference between the input image signal andthe image signal from the frame memory, whereby an image signal can beprovided free from higher frequency components in the encoder leading toa highly efficient coding.

Another advantageous feature is that filtering operation is designed todeal with any image signals comprehensively including pictures with noor little motion and pictures with motion. Therefore pictures withmotion can keep the original definition, while pictures with no orlittle motion can acquire high definition coded image by eliminatinghigher frequency components in the signal.

Another advantageous feature is that filtering can produce a highlycoding efficient image signal by eliminating higher frequency componentsin the signal in the event that the predictive signal still contains alarge amount of quantization error.

Another advantageous feature is that filtering intensity can be alteredin various degrees adapted to each image signal for a proper eliminationof higher frequency components in the signal. This can provide a highquality predictive image for a highly efficient coding with a smallercoding loop gain.

Another advantageous feature is that a pixel difference is used tocontrol coding error, so that a decoded image can carry less codederror. This can contribute to a highly efficient prediction for theforthcoming frames of an image signal.

As stated hereinbefore, the Interframe coding system and method of thepresent invention also have the following advantageous effects.

One of the advantageous features is that the coding controller outputsthe coding control signal based upon the "Activities" of the input imagesignal and the predictive error signal. This can improve codingefficiency.

Another advantageous feature is that the coding controller adds weighton the "Activity" of an input image signal. The coding controller cancontrol the output ratio of an image signal as the select signal fromthe coding controller in accordance with a user's or system's needs orthe condition of the processing image signal.

Another advantageous feature is that filtering control and codingcontrol are designed to deal with any image signals comprehensivelyincluding pictures with no or little motion and pictures with motion.Therefore pictures with motion can keep the original definition,

Awhile pictures with no or little motion can acquire high definitioncoded image by eliminating higher frequency components in the signal.

Another advantageous feature is that filtering can produce a highlycoding efficient image signal by eliminating higher frequency componentsin the signal in the event that the predictive signal still contains alarge amount of quantization error. Further, the coding controlleroutputs the coding control signal based upon the signal with thesmallest "Activity" among others, so that the coding efficiency can beimproved.

Another advantageous feature is that the coding controller outputs thecoding control signal based upon the mode signal by selecting the imagesignal with the smallest "Activity" among others. This can improve thecoding efficiency by providing a highly efficient prediction with lesscoding loop gain.

What is claimed is:
 1. A coding system comprising:an encoder, responsiveto a control signal, that selectively chooses a quantization step sizeof the encoder and that encodes an input signal into a coded signalusing the quantization step size; a frame memory that stores a previoussignal and provides a predicative signal corresponding to a differencebetween the input signal and the previous signal; means for generating apredictive error signal corresponding to a difference between the inputsignal and the predictive signal; a filter, having an input coupled tothe frame memory to receive the predictive signal, having a controlinput coupled to a filter control signal, and having an output coupledto an input of the means for generating a predictive error signal, thatreceives and filters the predictive signal, and that outputs a filteredsignal to the means for generating a predictive error signal; and afilter controller, having an output coupled to the control input of thefilter, that is responsive to the input signal and the predictivesignal, and that generates the filter control signal to control thefilter; and coding controller, responsive to the input signal and thepredictive error signal, that calculates characteristics of the inputsignal and the predictive error signal and that generates the codingcontrol signal the encoder based on the characteristics of the inputsignal and the predictive error signal.
 2. The coding system of claim 1,wherein the coding controller includes means for generating the codingcontrol signal based on the coded signal output by the encoder.
 3. Thecoding system of claim 1, wherein the input signal comprises anintraframe signal and the predictive error signal comprises aninterframe signal, and wherein the coding controller includes:means fordetermining characteristics of the interframe signal and the intraframesignal; means for comparing the characteristics of the interframe signaland the intraframe signal to produce a comparison result; and means forgenerating the coding control signal based on the comparison result. 4.The coding system of claim 3, wherein the comparing means includes meansfor weighting the characteristics of the interframe signal and theintraframe signal.
 5. The coding system of claim 3, wherein the meansfor determining characteristics determines a difference between maximumand minimum values of the signal.
 6. The coding system of claim 3,wherein the means for determining characteristics determines a sum ofabsolute difference values between individual values of a signal and amean value of the signal.
 7. The coding system of claim 3 wherein themeans for determining characteristics determines a sum of squareddifference values between individual values of a signal and a mean valueof a signal.
 8. The coding system of claim 1 wherein the codingcontroller receives a first interframe signal based on the filteredsignal, a second interframe signal based on the predictive signal, andan interframe signal corresponding to the input signal, the codingcontroller comprising;means for determining characteristics of the firstand second interframe signals and the intraframe signal; means forcomparing the characteristics of the first interframe signal, thecharacteristics of the second interframe signal, and the characteristicsof the interframe signal to produce a comparison result; and means forgenerating the coding control signal based on the comparison results. 9.The coding system of claim 8, wherein the comparing means includes meansfor weighting at least one of the characteristics of at least one of thefirst interframe signal, the second interframe signal, and theintraframe signal.
 10. The coding system of claim 8, wherein the meansfor determining characteristics determines a difference between maximumand minimum values of the signal.
 11. The coding system of claim 8,wherein the means for determining characteristics determines a sum ofabsolute difference values between individual values of a signal and amean value of the signal.
 12. The coding system of claim 8 wherein themeans for determining characteristics determines a sum of squareddifference values between individual values of a signal and a mean valueof the signal.
 13. The coding system of claim 1 wherein the codingcontroller receives an intraframe signal corresponding to the inputsignal and an interframe signal based on the filtered signal and whereinthe coding controller comprises:means for determining characteristics ofthe intraframe signal and characteristics of the intraframe signal;means for comparing the characteristics of the interframe signal withthe characteristics of the intraframe signal to provide a comparisonresult; and means for generating the coding control signal based on thecomparison result.
 14. The coding system of claim 13, wherein thecomparing means includes means for weighting at least one of thecharacteristics of at least one of the interframe signal and theintraframe signal.
 15. The coding system of claim 13, wherein the meansfor determining characteristics determines a difference between maximumand minimum values of a signal.
 16. The coding system of claim 13,wherein the means for determining characteristics determines a sum ofabsolute difference values between individual values of a signal and amean value.
 17. The coding system of claim 13 wherein the means fordetermining characteristics determines a sum of squared differencevalues between individual values of the signal and a mean value.